United States Patent and Trademark Office 



UNITED STATES DEPARTMENT OF COMMERCE 
United States Patent and Trademark Office 

Address: COMMISSIONER FOR PATENTS 
P.O. Box 1450 

Alexandria, Virginia 22313-1450 
www.uspto.gov 



APPLICATION NO. 


FILING DATE 


FIRST NAMED INVENTOR 


ATTORNEY DOCKET NO. 


CONFIRMATION NO. 


10/597,168 


07/31/2008 


Naotaka Ando 


3714449-00010 


7075 



24573 7590 

K&L Gates LLP 
P.O. Box 1135 
CHICAGO, IL 60690 



09/02/2010 



EXAMINER 



DANNEMAN, PAUL 



PAPER NUMBER 



3627 



NOTIFICATION DATE 



DELIVERY MODE 



09/02/2010 



ELECTRONIC 



Please find below and/or attached an Office communication concerning this application or proceeding. 

The time period for reply, if any, is set in the attached communication. 

Notice of the Office communication was sent electronically on above-indicated "Notification Date" to the 
following e-mail address(es): 

chicago.patents@klgates.com 



PTOL-90A (Rev. 04/07) 



\JI 1 1 Ot? n\* If SJtl OUffffffCff y 


Application No. 

10/597,168 


Applicant(s) 

AN DO, NAOTAKA 


Examiner 

PAUL DANNEMAN 


Art Unit 

3627 





- The MAILING DATE of this communication appears on the cover sheet with the correspondence address - 
Period for Reply 



A SHORTENED STATUTORY PERIOD FOR REPLY IS SET TO EXPIRE 3 MONTH(S) OR THIRTY (30) DAYS, 
WHICHEVER IS LONGER, FROM THE MAILING DATE OF THIS COMMUNICATION. 

- Extensions of time may be available under the provisions of 37 CFR 1.136(a). In no event, however, may a reply be timely filed 
after SIX (6) MONTHS from the mailing date of this communication. 

- If NO period for reply is specified above, the maximum statutory period will apply and will expire SIX (6) MONTHS from the mailing date of this communication. 

- Failure to reply within the set or extended period for reply will, by statute, cause the application to become ABANDONED (35 U.S.C. § 133). 
Any reply received by the Office later than three months after the mailing date of this communication, even if timely filed, may reduce any 
earned patent term adjustment. See 37 CFR 1 .704(b). 

Status 

1 )^l Responsive to communication(s) filed on 13 July 2006 . 
2a )□ This action is FINAL. 2b)^ This action is non-final. 

3) Q Since this application is in condition for allowance except for formal matters, prosecution as to the merits is 

closed in accordance with the practice under Ex parte Quayle, 1935 CD. 11, 453 O.G. 213. 

Disposition of Claims 

4) ^ Claim(s) 1-33 is/are pending in the application. 

4a) Of the above claim(s) is/are withdrawn from consideration. 

5) Q Claim(s) is/are allowed. 

6) |EI Claim(s) 1-33 is/are rejected. 

7) \Z\ Claim(s) is/are objected to. 

8) D Claim(s) are subject to restriction and/or election requirement. 

Application Papers 

9) D The specification is objected to by the Examiner. 

10) ^ The drawing(s) filed on 13 July 2006 is/are: a)£3 accepted or b)^ objected to by the Examiner. 

Applicant may not request that any objection to the drawing(s) be held in abeyance. See 37 CFR 1.85(a). 
Replacement drawing sheet(s) including the correction is required if the drawing(s) is objected to. See 37 CFR 1.121(d). 

1 1) 0 The oath or declaration is objected to by the Examiner. Note the attached Office Action or form PTO-152. 

Priority under 35 U.S.C. § 119 

12) ^ Acknowledgment is made of a claim for foreign priority under 35 U.S.C. § 1 19(a)-(d) or (f). 
a)|EI All b)Q Some * c)Q None of: 

1 Certified copies of the priority documents have been received. 

20 Certified copies of the priority documents have been received in Application No. . 

3.D Copies of the certified copies of the priority documents have been received in this National Stage 
application from the International Bureau (PCT Rule 17.2(a)). 
* See the attached detailed Office action for a list of the certified copies not received. 



Attach ment(s) 

1 ) ^ Notice of References Cited (PTO-892) 

2) Q Notice of Draftsperson's Patent Drawing Review (PTO-948) 

3) ^| Information Disclosure Statement(s) (PTO/SB/08) 

Paper No(s)/Mail Date . 



4) O Interview Summary (PTO-41 3) 

Paper No(s)/Mail Date. . 

5) O Notice of Informal Patent Application 

6) □ Other: . 



U.S. Patent and Trademark Office 

PTOL-326 (Rev. 08-06) 



Office Action Summary 



Part of Paper No./Mail Date 20100826 



Application/Control Number: 10/597,168 Page 2 

Art Unit: 3627 

DETAILED ACTION 
Status of the Claims 

1 . This Office Action is in response to the Application filed on 1 3 July 2006. 

2. Claims 1-33 are pending and have been examined in this Office Action. 

Claim Rejections - 35 USC §103 

3. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all obviousness 
rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as 
set forth in section 102 of this title, if the differences between the subject matter sought to be 
patented and the prior art are such that the subject matter as a whole would have been obvious 
at the time the invention was made to a person having ordinary skill in the art to which said 
subject matter pertains. Patentability shall not be negatived by the manner in which the invention 
was made. 

4. The factual inquiries set forth in Graham v. John Deere Co., 383 U.S. 1, 148 USPQ 459 (1966), 
that are applied for establishing a background for determining obviousness under 35 U.S.C. 103(a) are 
summarized as follows: 

1 . Determining the scope and contents of the prior art. 

2. Ascertaining the differences between the prior art and the claims at issue. 

3. Resolving the level of ordinary skill in the pertinent art. 

4. Considering objective evidence present in the application indicating obviousness or 
nonobviousness. 

5. Claims 1-9 and 12-15 are rejected under 35 U.S.C. 103(a) as being unpatentable over Du et al., 
US 7,734,729 B2 ("Du"). 



Claim 1: 

With regard to the system limitations: 
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Du in at least Fig. 1 , Fig. 3 and Column 4, lines 1-15 discloses an information retrieving system. 
Du in at least Fig.4A Column 2, lines 1-31 discloses a portable imaging device (mobile phone, 
digital camera, PDA and etc.) for capturing imaging data of a desired item. The imaging data is 
communicated to one or more servers. Identifying data may be extracted from the image using 
any one of the standard image recognition and processing programs available. The data 
identifying the selected item is then used to query one or more resources to obtain item 
information related to the selected item. The obtained information is then communicated to the 
mobile phone (portable imaging device) for display to the consumer. 

Du in at least Fig.2A, Column 7, lines 59-67, Column 8, lines 1-14 and lines 65-67 and Column 9, 
lines 1-24 discloses that the images may contain graphic design, such as a symbol or trademark 
and are stored in any one of several databases used to identify the selected item. 

Claims 2, 3 and 4: 

With regard to the further limitation of Claim 1: 

Du in at least Column 3, lines 31-67 further discloses that image captured by the portable imaging 
device and communicated to a server is analyzed and data identifying the selected item is 
extracted from the image . The identifying data may include Universal Product Code (UPC), text 
on the packaging of the product, indicia on the product itself, etc. Du in at least Column 7, lines 
5-16 further discloses that the image captured may be that of the product itself. 

Claims 5-8: 

With regard to the further limitation of Claim 1: 

Du in at least Fig. 3, Column 7, lines 26-67 and Column 8, lines 1-39 discloses that the image may 
comprise either a gray scale image or a color image . The image may also include full-motion 
video images in addition to, or instead of, a still image. Du in at least Column 8, lines 40-64 
further discloses that identifying data may be extracted from the captured image either before or 
after the image is communicated to the server. 
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Claim 9: 

With regard to the further limitation of Claim 1: 

Du in at least Fig. 3 and Column 6, lines 57-67 discloses that the information processing method 
300 process an image to identify the item identified by the image and returns to the user 
information pertaining to the identified item. 

Claims 12-14: 

With regard to the further limitation of Claim 1: 

Du in at least Fig. 3 and Column 6, lines 57-67 discloses that the information processing method 
300 process an image to identify the item identified by the image and return to the user 
information pertaining to the identified item. 

Claim 15: 

With regard to the further limitation of Claim 1: 

Du in at least Fig.4A Column 2, lines 1-31 discloses a portable imaging device (mobile phone, 
digital camera, PDA and etc.) for capturing imaging data of a desired item. 

6. Claims 10-11 are rejected under 35 U.S.C. 103(a) as being unpatentable over Du et al., US 
7,734,729 B2 ("Du") as applied to claims 1-9 above, and further in view of Ogasawara, US 6,512,919 B2 
and further in view of Lennon, US 6,624,843 B2. 

Claims 10 and 11: 

With regard to the further limitation of Claim 9: 

Du in at least Column 3, lines 31-67 further discloses that image captured by the portable imaging 
device and communicated to a server is analyzed and data identifying the selected item is 
extracted from the image . The identifying data may include Universal Product Code (UPC), text 
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on the packaging of the product, indicia on the product itself, etc. Du in at least Column 7, lines 
5-16 further discloses that the image captured may be that of the product itself. 
Du does not specifically disclose recognizing a shape; however Ogasawara in at least Column 2, 
lines 60-67 and Column 3, lines 1-3 discloses an electronic shopping system for facilitating 
purchase transactions via a wireless videophone. Ogasawara in at least Column 3, lines 4-20 
further discloses that the electronic shopping system comprises a server and at least one 
wireless videophone for communicating with the server. Ogasawara in at least Column 22, lines 
59-67 and Column 23, lines 1-10 further discloses that the present invention allows bar code 
and/or alpha-numeric information to be captured. Ogasawara further discloses that the pattern 
and/or character recognition and data processing may be performed within the video phone or at 
the server that receives the image data from the video phone. 

Ogasawara in at least Column 23, lines 11-30 further discloses that advanced pattern recognition 
software may be used to extend the capabilities of the image recognition software to include 
identifying any merchandise item having a distinct or identifiable shape or other visually 
identifiable characteristics and returning the merchandise information to the customer for display 
on the wireless video phone. 

It would have been obvious, at the time of the invention to combine the well known elements of 
Du's system and method for obtaining information related to an item using a portable imaging 
device with the equally well known features of Ogasawara's advanced pattern recognition 
software, by known methods with no change in their respective functions, and where the 
combination would yield predictable results. 

Du and Ogasawara do not disclose recognizing a person; however Lennon in at least Colum 1, 
lines 13-16 discloses an invention related to commercial system having a method and apparatus 
for capturing a person's image and using the captured image in a retailing system. 
Lennon in at least Column 2, lines 29-51 further discloses that the computer system is capable of 
merging video or still images of live customers with video or still images of stored reference 
model images wearing an apparel item. The computer system retrieves the stored reference 
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images from a database. Lennon in at least Column 5, lines 52-67 further discloses the use of 
storage programs and visual pattern recognition programs which are used to create the database 
of items with the meta information that describes each reference image. 

It would have been obvious, at the time of the invention to modify the Du and Ogasawara 
combination with Lennon's visual pattern recognition program to recognize a person, with the 
motivation to enhance the shopping system's ability to recognize people, characters and shapes 
and make a customer's shopping experience memorable. 

7. Claims 16-19, 20-26 and 31-33 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Du et al., US 7,734,729 B2 ("Du") as applied to claims 1-9 above, and further in view of Lennon, US 
6,624,843 B2. 

Claims 16-19: 

With regard to the system limitations: 

Du in at least Fig. 1 , Fig. 3 and Column 4, lines 1-15 discloses an information retrieving system. 
Du in at least Fig.4A Column 2, lines 1-31 discloses a portable imaging device (mobile phone, 
digital camera, PDA and etc.) for capturing imaging data of a desired item. The imaging data is 
communicated to one or more servers. Identifying data may be extracted from the image using 
any one of the standard image recognition and processing programs available. The data 
identifying the selected item is then used to guery one or more resources to obtain item 
information related to the selected item. The obtained information is then communicated to the 
mobile phone (portable imaging device) for display to the consumer. 

Du in at least Fig.2A, Column 7, lines 59-67, Column 8, lines 1-14 and lines 65-67 and Column 9, 
lines 1-24 discloses that the images may contain graphic design, such as a symbol or trademark 
and are stored in any one of several databases used to identify the selected item. 
Du does not specifically disclose the use of meta data (meta information); however Lennon in at 
least Column 2, lines 29-51 further discloses that the computer system is capable of merging 
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video or still images of live customers with video or still images of stored reference model images 
wearing an apparel item. The computer system retrieves the stored reference images from a 
database. Lennon in at least Column 5, lines 52-67 further discloses the use of storage programs 
and visual pattern recognition programs which are used to create the database of items with the 
meta information that describes each reference image. 

It would have been obvious, at the time of the invention to modify the Du's system and method for 
obtaining information related to an item of commerce using a portable imaging device with 
Lennon's visual pattern recognition program which is used to create the database of items with 
the meta information that describes each reference image in the database with the motivation to 
provide a shopping system with as item information that helps a customer decide on purchasing 
an item. 

Claims 20 and 21: 

With regard to the image recognizing apparatus limitations: 

Du in at least Fig. 1 , Fig. 3 and Column 4, lines 1-15 discloses an information retrieving system. 
Du in at least Fig.4A Column 2, lines 1-31 discloses a portable imaging device (mobile phone, 
digital camera, PDA and etc.) for capturing imaging data of a desired item. The imaging data is 
communicated to one or more servers. Identifying data may be extracted from the image using 
any one of the standard image recognition and processing programs available. The data 
identifying the selected item is then used to query one or more resources to obtain item 
information related to the selected item. The obtained information is then communicated to the 
mobile phone (portable imaging device) for display to the consumer. 

Du in at least Fig.2A, Column 7, lines 59-67, Column 8, lines 1-14 and lines 65-67 and Column 9, 
lines 1-24 discloses that the images may contain graphic design, such as a symbol or trademark 
and are stored in any one of several databases used to identify the selected item. 
Du does not specifically disclose the use of meta data; however Lennon in at least Column 5, 
lines 52-67 further discloses the use of storage programs and visual pattern recognition programs 
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which are used to create the database of items with the meta information that describes each 
reference image. 

It would have been obvious, at the time of the invention to modify the Du's system and method for 
obtaining information related to an item of commerce using a portable imaging device with 
Lennon's visual pattern recognition program which is used to create the database of items with 
the meta information that describes each reference image in the database with the motivation to 
provide a shopping system with item information that describes an item and assists the customer 
in determining if this is the item they are searching for and may wish to purchase. 

Claims 22-25: 

With regard to the further limitation of Claim 20: 

Du in at least Fig. 3, Column 7, lines 26-67 and Column 8, lines 1-39 discloses that the image may 
comprise either a gray scale image or a color image . The image may also include full-motion 
video images in addition to, or instead of, a still image. Du in at least Column 8, lines 40-64 
further discloses that identifying data may be extracted from the captured image either before or 
after the image is communicated to the server. 

Claim 26: 

With regard to the further limitation of Claim 20: 

Du in at least Fig. 3 and Column 6, lines 57-67 discloses that the information processing method 
300 process an image to identify the item identified by the image and returns to the user 
information pertaining to the identified item. 

Claims 31-33: 

With regard to the sales system limitations: 
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Du in at least Column 2, lines 3-48 discloses an interactive merchandising system for 

communication information associated with commerce items, e.g., price, availability, reviews, etc. 

provided by e.g., a brick-and-mortar retail store and/or an online retail store . 

Du in at least Column 3, lines 31-67 further discloses that image captured by the portable imaging 

device and communicated to a server is analyzed and data identifying the selected item is 

extracted from the image . The identifying data may include Universal Product Code (UPC), text 

on the packaging of the product, indicia on the product itself, etc. Du in at least Column 7, lines 

5-16 further discloses that the image captured may be that of the product itself. 

Du in at least Column 4, lines 16-47 discloses that the server 103 may be a computer that is 

associated with a store or other entity that provides goods and/or services to others, whether 

retail, wholesale or otherwise, or any other entity. 

Du in at least Column 10, lines 40-46 further discloses that when item information is 
communicated back to the user's mobile device, the user may be provided with a control that 
allows the user to immediately purchase the selected item from the online source. Existing 
software configured to execute electronic commerce purchase transactions can be used to 
implement such a feature. 

Du does not disclose the use of meta data; however Lennon in at least Colum 1, lines 13-16 
discloses an invention related to commercial system having a method and apparatus for 
capturing a person's image and using the captured image in a retailing system. 
Lennon in at least Column 2, lines 29-51 further discloses that the computer system is capable of 
merging video or still images of live customers with video or still images of stored reference 
model images wearing an apparel item. The computer system retrieves the stored reference 
images from a database. Lennon in at least Column 5, lines 52-67 further discloses the use of 
storage programs and visual pattern recognition programs which are used to create the database 
of items with the meta information that describes each reference image. 

It would have been obvious, at the time of the invention to modify the Du's system and method for 
obtaining information related to an item of commerce using a portable imaging device for 
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conducting purchases with Lennon's visual pattern recognition program to recognize a person 
and use of meta data, with the motivation to enhance the shopping system's ability to locate the 
items they are searching for and purchasing those items. 

8. Claims 27-28 and 29-30 are rejected under 35 U.S.C. 103(a) as being unpatentable over Du et 
al., US 7,734,729 B2 ("Du") and further in view of Lennon, US 6,624,843 B2 as applied to claim 26 above, 
and further in view of Ogasawara, US 6,512,919 B2. 

Claims 27 and 28: 

With regard to the further limitation of Claim 26: 

Du in at least Column 3, lines 31-67 further discloses that image captured by the portable imaging 
device and communicated to a server is analyzed and data identifying the selected item is 
extracted from the image . The identifying data may include Universal Product Code (UPC), text 
on the packaging of the product, indicia on the product itself, etc. Du in at least Column 7, lines 
5-16 further discloses that the image captured may be that of the product itself. 
Du does not disclose recognizing a person; however Lennon in at least Colum 1, lines 13-16 
discloses an invention related to commercial system having a method and apparatus for 
capturing a person's image and using the captured image in a retailing system. 
Lennon in at least Column 2, lines 29-51 further discloses that the computer system is capable of 
merging video or still images of live customers with video or still images of stored reference 
model images wearing an apparel item. The computer system retrieves the stored reference 
images from a database. Lennon in at least Column 5, lines 52-67 further discloses the use of 
storage programs and visual pattern recognition programs which are used to create the database 
of items with the meta information that describes each reference image. 

It would have been obvious, at the time of the invention to modify the Du's system and method for 
obtaining information related to an item of commerce using a portable imaging device with 
Lennon's visual pattern recognition program to recognize a person, with the motivation to 
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enhance the shopping system's ability to recognize people, characters and shapes and make a 
customer's shopping experience memorable. 

Du and Lennon do not specifically disclose recognizing a shape; however Ogasawara in at least 
Column 2, lines 60-67 and Column 3, lines 1-3 discloses an electronic shopping system for 
facilitating purchase transactions via a wireless videophone. Ogasawara in at least Column 3, 
lines 4-20 further discloses that the electronic shopping system comprises a server and at least 
one wireless videophone for communicating with the server. Ogasawara in at least Column 22, 
lines 59-67 and Column 23, lines 1-10 further discloses that the present invention allows bar code 
and/or alpha-numeric information to be captured. Ogasawara further discloses that the pattern 
and/or character recognition and data processing may be performed within the video phone or at 
the server that receives the image data from the video phone. 

Ogasawara in at least Column 23, lines 11-30 further discloses that advanced pattern recognition 
software may be used to extend the capabilities of the image recognition software to include 
identifying any merchandise item having a distinct or identifiable shape or other visually 
identifiable characteristics and returning the merchandise information to the customer for display 
on the wireless video phone. 

It would have been obvious, at the time of the invention to combine the well known elements of 
Du's system and method for obtaining information related to an item using a portable imaging 
device as modified with Lennon's visual pattern recognition program to recognize a person with 
the equally well known features of Ogasawara's advanced pattern recognition software, by known 
methods with no change in their respective functions, and where the combination would yield 
predictable results. 

Claims 29 and 30: 

With regard to the image recognizing apparatus limitations: 

Du in at least Fig. 1 , Fig. 3 and Column 4, lines 1-15 discloses an information retrieving system. 
Du in at least Fig.4A Column 2, lines 1-31 discloses a portable imaging device (mobile phone, 
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digital camera, PDA and etc.) for capturing imaging data of a desired item. The imaging data is 
communicated to one or more servers. Identifying data may be extracted from the image using 
any one of the standard image recognition and processing programs available. The data 
identifying the selected item is then used to query one or more resources to obtain item 
information related to the selected item. The obtained information is then communicated to the 
mobile phone (portable imaging device) for display to the consumer. 

Du in at least Fig.2A, Column 7, lines 59-67, Column 8, lines 1-14 and lines 65-67 and Column 9, 
lines 1-24 discloses that the images may contain graphic design, such as a symbol or trademark 
and are stored in any one of several databases used to identify the selected item. 
Du does not specifically disclose the use of meta data; however Lennon in at least Column 5, 
lines 52-67 further discloses the use of storage programs and visual pattern recognition programs 
which are used to create the database of items with the meta information that describes each 
reference image. 

It would have been obvious, at the time of the invention to modify the Du's system and method for 
obtaining information related to an item of commerce using a portable imaging device with 
Lennon's visual pattern recognition program which is used to create the database of items with 
the meta information that describes each reference image in the database with the motivation to 
provide a shopping system with item information that describes an item and assists the customer 
in determining if this is the item they are searching for and may wish to purchase. 
Du and Lennon do not specifically disclose recognizing a shape; however Ogasawara in at least 
Column 2, lines 60-67 and Column 3, lines 1-3 discloses an electronic shopping system for 
facilitating purchase transactions via a wireless videophone. Ogasawara in at least Column 3, 
lines 4-20 further discloses that the electronic shopping system comprises a server and at least 
one wireless videophone for communicating with the server. Ogasawara in at least Column 22, 
lines 59-67 and Column 23, lines 1-10 further discloses that the present invention allows bar code 
and/or alpha-numeric information to be captured. Ogasawara further discloses that the pattern 
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and/or character recognition and data processing may be performed within the video phone or at 
the server that receives the image data from the video phone. 

Ogasawara in at least Column 23, lines 11-30 further discloses that advanced pattern recognition 
software may be used to extend the capabilities of the image recognition software to include 
identifying any merchandise item having a distinct or identifiable shape or other visually 
identifiable characteristics and returning the merchandise information to the customer for display 
on the wireless video phone. 

It would have been obvious, at the time of the invention to combine the well known elements of 
Du's system and method for obtaining information related to an item using a portable imaging 
device as modified with Lennon's use of meta information in a database with the equally well 
known features of Ogasawara's advanced pattern recognition software, by known methods with 
no change in their respective functions, and where the combination would yield predictable 
results. 

Conclusion 

9. Any inquiry concerning this communication or earlier communications from the examiner should 
be directed to PAUL DANNEMAN whose telephone number is (571)270-1863. The examiner can 
normally be reached on Mon.-Thurs. 6AM-5PM Fri. off. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's supervisor, 
Florian Zeender can be reached on 571-272-6790. The fax phone number for the organization where this 
application or proceeding is assigned is 571-273-8300. 
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Information regarding the status of an application may be obtained from the Patent Application 
Information Retrieval (PAIR) system. Status information for published applications may be obtained from 
either Private PAIR or Public PAIR. Status information for unpublished applications is available through 
Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) 
at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative 
or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272- 
1000. 

/Paul Danneman/ 
Examiner, Art Unit 3627 
27 August 2010 



IF. Ryan Zeender/ 
Supervisory Patent Examiner, Art Unit 3627 



